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Pretesting  Is  Essential;  You  Can  Choose 
From  Various  Methods 

Use  pretesting  to  help  design  materials  and  messages  that  work. 
Here  are  six  methods  from  which  to  choose. 


w 

\ 


Pretests  show  what  works  with  a  particular 
audience.    The  slogan  that  seems  persua- 
sive to  an  adult  may  be  confusing  to  a 
sixth-grader.    A  poster  that  captures  the 

attention  of 


teenagers  in 
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other  drugs 

to  a  program's 

target  audience.    Specifically,  pretesting  can 
help  program  managers: 

■  Select  message  concepts  —  styles, 
formats,  spokesperson,  and  appeals 
(such  as  fear,  humor,  compassion) 

■  Guide  creative  work 


■  Fine  tune  wording  and  visual  images 

■  Guide  revisions  (before  spending 
time  and  money  on  the  finished 
product). 

Pretesting  is  valuable  at  several  stages 
of  message  and  material  development. 
Some  methods  can  be  used  in  the  early 
stages  to  test  concepts  or  general  issues 
and  to  spark  ideas;  other  methods  are 
more  useful  when  materials  are  in 
close-to-final  form. 

Factors  to  consider  in  selecting  a  pretest 
method  include,  in  addition  to  the  stage 
of  development  of  the  materials,  the 
kind  of  audience  at  which  the  materials 
are  aimed  (e.g.,  professional,  rural),  the 
sensitivity  and  complexity  of  the  materi- 
als, and  the  resources  available.    The 
following  overview  summarizes  the 
purposes,  requirements,  pros,  and 
cons  of  six  pretest  methods. 


1.  Focus  Groups 

Purpose:   To  obtain  insights  about 
audience  concerns,  beliefs,  reactions, 
and  vocabulary. 

Focus  groups  are  small  gatherings  of 
8  to  12  people  who  meet  with  a  trained 
moderator  to  talk  about  ideas  and 
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Before  You  Begin... 


Don't  plunge  in  until  you've  considered  several  issues: 

■  Pretesting  is  just  one  step  in  the  communications  process.    Before  you  begin  to  plan 
materials  and  messages,  you  must  define  your  target  audience,  determine  their  needs, 
and  spell  out  your  own  goals  and  objectives. 

See  the  Resources  list  on  pages  5  and  6  for  possible  sources  of  information  on  health 
communications . 

■  These  sources  can  help  you  find  out  what  other  materials  exist  for  your  target 
audience  and  whether  they  have  been  pretested.    One  call  to  a  resource  center  may 
turn  up  this  kind  of  information. 

■  Valuable  as  pretesting  is,  it  has  the  limitations  of  qualitative  research;  that  is,  it 
provides  insights  and  ideas.    It  usually  cannot  provide  statistically  valid  conclusions 
about  your  entire  audience.    Not  enough  people  are  involved,  nor  have  they  been 
randomly  selected.    Use  quantitative  research,  such  as  polls  and  surveys,  to  obtain 
statistically  valid  data. 


materials.    This  pretesting  method  is  especially 
useful  to  answer  questions  in  the  early  stages 
of  a  project:    What  kind  of  spokesperson  could 
convince  sixth-graders  that  drugs  are  not  the 
path  to  popularity?   What  kind  of  format  would 
parents  prefer  for  information  about  alcohol, 
tobacco,  and  other  drugs?     Also  consider 
focus  groups  whenever  you  need  new  ideas 
and  insights,  which  can  emerge  from  group 
interaction. 

For  example:   To  help  develop  an  antialcohol 
campaign  slogan,  preteens  were  asked  to 
choose  between  six  alternatives.    On  a 
questionnaire,  one  slogan  emerged  as  the 
clear  favorite.    But  in  subsequent  focus  groups 
with  the  children,  another  slogan  —  "Drinking 
hurts"  —  appeared  to  evoke  deeper  feelings  and 
led  to  lively  discussions.    As  a  result,  planners 
learned  that  this  concept  could  also  be  an 
important  component  of  campaign  materials. 

Use  to  pretest:  Program  themes  and  images, 
general  issues,  materials  in  early  stage  of 
development. 

Minimum  number  of  respondents:  8  to 

12  per  group;  at  least  two  groups  per  type 
of  respondent. 


Resources  needed:  Trained  moderator, 
discussion  outline,  list  of  respondents,  fees 
or  incentives  to  participate,  meeting  room, 
recording  equipment,  AV  equipment  if 
pretesting  audiovisuals,  analysis  expertise. 

Pros:  Group  interaction  stimulates  responses; 
can  cover  many  topics. 

Cons:  Individual  responses  may  be  swayed  by 
group  response;  can  provide  misleading  informa- 
tion if  not  skillfully  managed;  cannot  probe 
sensitive  or  complex  questions. 


2.  Individual  Interviews 

Purpose:    To  explore  individuals'  responses, 
feelings,  and  concerns. 

Individual  interviews  elicit  reactions  to  sensitive 
issues  or  complex  materials  and  permit  lengthy 
discussion  of  draft  materials.    Interviews  are 
worth  the  extra  time  and  expense  when  the 
response  of  an  individual,  free  of  group  pres- 
sure, is  important,  and  when  long,  confidential 
conversations  may  provide  special  insights. 
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For  example:   A  coloring  book  for  children 
of  alcoholics  could  be  pretested  in  individual 
interviews  to  help  understand  the  children's 
responses.    Certain  images  in  the  coloring  book 
might  be  so  strong  that  the  child's  response  is 
denial  that  they  apply  to  him  or  her.    A  quiet 
talk  with  a  trained  interviewer  can  give  the  child 
time  to  express  complex  feelings  and  provide 
ideas  for  less  threatening  images  that  might  be 
more  useful  in  reaching  this  audience. 

Use  to  pretest:  Program  themes  and  images, 
sensitive  issues,  complex  materials. 

Minimum  number  of  respondents:  10  per 

type  of  respondent. 

Resources  needed:  List  of  potential  respon- 
dents, fees  or  incentives  (optional),  trained 
interviewer,  telephone  or  quiet  room,  tape 
recorder  (optional),  analysis  expertise. 

Pros:  Can  contact  hard-to-reach  audiences, 
such  as  people  with  limited  reading  and  writing 
skills;  can  explore  emotional  or  complex  issues. 

Cons:  Can  be  time  consuming  to  conduct  and 
analyze;  may  yield  no  patterns  in  responses, 
especially  if  number  of  interviews  is  limited. 


3.  Central  Location  Intercept 
Interviews 

Purpose:    To  obtain  many  individual  reactions. 

Intercept  interviews  take  places,  such  as  shop- 
ping malls  and  schools,  where  many  members 
of  a  target  audience  congregate.    Intercept 
interviewers  use  questionnaires  designed  for 
quick  answers,  so  that  each  interview  is  short 
and  the  results  can  be  tabulated  easily.    Use 
intercept  interviews  when  the  stimulus  of 
group  interaction  is  not  important. 

For  example:   To  pretest  a  poster  urging 
pregnant  Hispanic  women  not  to  use  alcohol, 
interviewers  can  be  stationed  at  a  laundromat  in 
a  predominantly  Hispanic  neighborhood.    The 
interviewers  approach  women  who  appear  to 
be  of  child-bearing  age.    They  explain  the 
purpose  of  the  pretest,  ask  a  few  screening 
questions  to  be  sure  the  women  belong  to  the 


target  audience,  and  then  ask  questions  about 
the  poster:    "Is  there  anything  you  particularly 
like  or  dislike  about  this  poster?"    "What  is  its 
main  idea?"    If  a  large  proportion  do  not 
perceive  the  main  idea  to  be  the  one  intended, 
the  poster's  language  can  be  clarified. 

Use  to  pretest:  Print  and  audiovisual  materials, 
program  themes  and  images. 

Minimum  number  of  respondents:  60  to 

100  per  target  audience. 

Resources  needed:  Trained  interviewer, 
structured  questionnaire,  access  to  central 
location  (e.g.,  school,  laundromat,  mall),  quiet 
spot  for  interview,  AV  equipment  if  pretesting 
audiovisuals;  analysis  expertise. 

Pros:  Is  quick;  can  be  inexpensive;  can  be 
designed  to  obtain  quantitative  data. 

Cons:  Short;  cannot  probe  sensitive  or 
complex  questions. 


4.  Theater  Testing 


Purpose:  To  obtain  many  responses  at  once; 
to  measure  recall  under  conditions  that  simu- 
late actual  viewing. 

These  pretests  gather  large  numbers  of  people 
together  to  view  messages  embedded  in  other 
programming.    This  method  simulates  actual 
viewing  conditions.    Theater  testing  can  help 
you  learn  how  and  whether  your  message 
stands  out  among  the  clutter  of  other  messages 
broadcast  each  day. 

For  example:   To  pretest  a  public  service 
announcement  (PSA)  aimed  at  teenagers,  a 
theater  test  in  a  classroom  can  show  it  as  part 
of  a  videotaped  television  program.    Students 
then  fill  out  a  questionnaire  asking  what  they 
remember  and  how  they  liked  the  program, 
other  commercials,  and  the  PSA.    If  responses 
show  that  the  PSA  was  not  well  remembered,  a 
more  dramatic  presentation  may  be  developed. 

Use  to  pretest:  Audiovisual  materials  (which 
can  then  be  adapted  to  print  materials  such  as 
posters  displayed  with  other  wall  hangings  in 
a  clinic). 


Minimum  number  of  respondents:  60  to 

100  per  target  audience. 

Resources  needed:  List  of  potential  respon- 
dents, fees  or  incentives  to  participate,  large 
room,  AV  equipment,  questionnaire,  analysis 
expertise. 

Pros:  Can  simulate  natural  exposure  to  materi- 
als and  help  gauge  how  they  compete  with 
other  messages;  permits  quick  analysis  of 
responses. 

Cons:  Can  be  time  consuming  to  arrange;  can 
be  expensive. 


5.  Self-Administered 
Questionnaires 

Purpose:  To  obtain  many  individuals'  reactions 
to  materials. 

Mailed  or  personally  delivered,  questionnaires 
accompany  draft  materials  and  elicit  brief, 
individual  responses.    Questionnaires  can  be 
useful  in  reaching  audiences  who  are  widely 
dispersed,  do  not 


Also  use 
questionnaires 
to  have 
materials 
reviewed  by 
"gatekeepers" — 
such  as  radio 
talk  show  hosts, 
clinic  directors, 
and  teachers. 


congregate  in  public 
places,  and  are  not 
likely  to  attend  focus 
groups,  such  as 
professionals  and 
people  living  in  rural 
areas. 

For  example:   Ques- 
tionnaires can  be  used 
to  learn  the  reactions 
of  school  principals  to 
proposed  materials. 
In  a  pretest  of  a 
sticker  to  be  placed 
on  book  bags  and 
lockers,  children  in 
focus  groups  said  they  thought  a  skull-and- 
cross-bones  image  was  "cool"  —  particularly 
after  group  pressure  began  to  have  its  effect. 
But  principals  who  were  surveyed  separately 
said  that  they  would  not  allow  that  particular 
sticker  to  be  distributed.    One  explained  that 


the  design  said  to  children  that  risky  behavior 
itself  was  "cool." 

Use  to  pretest:  Print  and  audiovisual  materials. 

Minimum  number  of  respondents:  20 

(100  to  200  is  ideal). 

Resources  needed:  List  of  potential  respon- 
dents, questionnaire,  postage  if  mailed,  AV 
equipment  if  pretesting  audiovisuals,  analysis 
expertise. 

Pros:  Is  inexpensive;  can  reach  homebound  or 
rural  or  other  dispersed  audiences  easily. 

Cons:  May  have  low  response  rate  (if  mailed); 
may  require  followup;  may  take  a  long  time  to 
receive  responses;  does  not  permit  control  of 
exposure  to  materials. 


6.  Readability  Testing 

Purpose:   To  gauge  the  reading  level  of 
materials. 

If  most  members  of  your  audience  read  at  the 
ninth-grade  level,  your  written  materials  should 
be  at  about  this  level.    Readability  formulas 
provide  a  rough  indication  of  the  grade  level 
needed  to  understand  text.    When  using  them, 
however,  remember  that  other  factors,  such 
as  graphics  and  type  size,  also  influence 
readability. 

For  example:   The  SMOG  formula,  one 
widely  used  method  to  test  readability,  uses 
both  sentence  length  and  syllable  counts  to 
estimate  reading  level.    According  to  the  SMOG 
formula,  many  health-related  articles  in  popular 
magazines  are  written  at  the  12th-grade  level 
and  higher. 

Use  to  pretest:  All  print  materials. 

Number  of  respondents:  None. 

Resources  needed:  A  readability  formula 
(see  Resources  list  on  pages  5  and  6). 

Pros:  Is  fast;  is  inexpensive. 

Cons:  Provides  only  a  rough  indication; 
does  not  obtain  audience  reaction. 
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Pretesting  With  Youth  at  Risk 


Be  aware  of   some  special  issues  when 
conducting  pretests  with  youth  in  high-risk 
environments.    These  include: 

■  Lack  of  trust.    Interviewers  may  be 
seen  as  people  in  positions  of  authority 
and  may  arouse  suspicion.    Children  and 
teenagers  are  quick  to  spot  any  sign  of 
falseness  and  feel  that  they  are  being 
used  or  manipulated. 

■  Need  to  test.    These  youngsters  may 
need  to  test  the  sincerity  and  "hipness" 
of  the  facilitator.    If  they  can  dupe  him 
or  her  into  believing  something,  they 
may,  as  a  group,  quickly  make  the 
pretest  into  a  game  of  deception,  thus 
turning  the  tables  on  the  perceived 
manipulator. 

■  Need  to  please.    High-risk  youth  may 
misinterpret  the  adult's  interest  as  a  form 
of  personal  attention  and  caring,  which 
may  be  absent  from  many  of  their 
encounters  with  adults.    As  a  result, 
they  may  become  anxious  to  please 

the  facilitator  by  giving  the  desired 
response. 

Here  are  ways  to  cope  with  these 
problems: 

■  Plan  to  hold  more  pretests  than  are 
normally  recommended 


Use  a  facilitator  who  shares 
some  of  the  characteristics  or 
background  of  the  target 
audience.    Or  use  two  interviewers, 
one  of  whom  is  familiar  with  the 
audience. 

Allow  ample  time  to  explain  the 
purpose  of  the  pretests  to 
participants.    Interviewers  should 
tell  how  the  information  will  be  used 
and  the  reasons  for  any  recording 
devices.    They  should  also 
emphasize  the  eventual  benefits 
of  the  materials  for  the  audience. 

Make  it  clear  that  there  are  no 
"right"  answers.    In  the  absence  of 
right  answers,  youngsters  may  feel 
uncomfortable  and,  in  a  group 
setting,  may  defer  to  a  natural  leader. 
By  making  it  very  clear  that  no 
answer  is  more  desirable  than  any 
other  and  by  responding  to  all 
comments  in  a  positive  way,  the 
interviewer  may  be  able  to  minimize 
this  problem. 

Have  the  results  reviewed  by 
someone  familiar  with  the  young 
people's  environment.    They  may 
have  some  special  insights,  especially 
into  group  interactions. 


Resources 

Help  with  pretesting  is  available  from  many 
sources.    University  departments  of  marketing, 
communications,  or  health  education  may  be 
able  to  help.    Marketing  research  firms  can  be 
hired  to  help  with  certain  aspects  of  pretesting 
or  to  conduct  the  entire  process.    For  help  in 
getting  started,  consider: 


American  Marketing  Association.   Marketing 
Services  Guide.    250  South  Wacker  Drive,  Suite 
200,  Chicago,  IL   60606,  (312)  648-0536.    This 
guide,  published  annually,  lists  suppliers  and 
services  all  over  the  United  States  and  is 
available  from  the  association. 

CSAP  Communications  Team,  7200  Wisconsin 
Avenue,  Suite  500,  Bethesda,  MD  20814-4820. 
See  box  on  back  cover. 


Marketing  Research  Association,  111  East 
Wacker  Drive,  Suite  600,  Chicago,  IL  60601, 
(312)  644-6610.  Publishes  audience  research 
guidelines  and  interviewer  training  materials. 

National  Clearinghouse  for  Alcohol  and  Drug 
Information,  P.O.  Box  2345,  Rockville,  MD 
20852,    (301)  468-2600  or  (800)  729-6686. 
Provides  information  on  research  literature, 
programs,  and  educational  materials.    Ask 
about  the  Regional  Alcohol  and  Drug  Aware- 
ness Resource  (RADAR)  network,  which  may 
have  a  resource  center  in  your  region. 

Office  of  Cancer  Communications, 
National  Cancer  Institute.   Making  Health 


Communication  Programs  Work:  A  Planners 
Guide:  1989.  This  guide  to  all  aspects  of  health 
communications,  including  pretests  is  available 
from  OCC,  Bethesda,  MD  20892,  (301)  496-5583 
or  (800)  422-6237.    This  guide  also  describes  the 
SMOG  readability  formula  and  includes  sample 
instruments  for  all  six  pretest  methods. 

Public  Relations  Society  of  America,  33  Irving 
Place,  3rd  Floor,  15th  and  16th  Streets,  New 
York,  NY    10003-    Ask  about  publications  and 
local  chapters'  pro  bono  projects. 

Rice,  Ronald  E.,  and  Atkin,  Charles  K.,  eds., 
Public  Communication  Campaigns.  Beverly 
Hills,  CA,  Sage,  1989. 
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Please  feel  free   to   be  a    "copy   cat"   and 
make   all   the   copies   you    want. 

You    have    our    permission! 


Developed  and  Produced  by  the  CSAP  Communications  Team. 

Patricia  A.  Wright,  Ed.D.,  Managing  Editor. 

Distributed  by  the  National  Clearinghouse  for  Alcohol  and  Drug  Information,  P.O.  Box  2345, 
Rockville,  MD  20852. 

This  bulletin  is  one  in  a  series  developed  to  assist  programs  that  are  working  to  prevent  alcohol, 
tobacco,  and  other  drug  problems.  We  welcome  your  suggestions  regarding  information  that  may 
be  included  in  future  bulletins.    For  help  in  learning  about  your  audience,  developing  messages 
and  materials,  and  evaluating  communication  programs,  contact  the  CSAP  Communications  Team, 
7200  Wisconsin  Avenue,  Suite  500,  Bethesda,  MD  20814-4820,  (301)  951-3277. 


